Design Considerations in Instrumenting and Monitoring Web-Based Information Retrieval Systems

The Internet Web environment opens up extraordinary information are changing rapidly. Daily, hundreds of opportunities for user access to information. Techniques thousands of individuals use the Internet and Web for monitoring users and systems and for evaluating sysbrowsers like Netscape to obtain a much broader range tem design and performance have not kept pace with of information than has previously been possible. This Web development. This article reviews concepts of Web information consists not only of bibliographic citations operations (including browsers, clients, information retrieval applications, servers, and data communications but of full text, data, images, sound, programs, and the systems) with specific attention given to how monitoring like. The manner of searching has changed as has the should take place and how privacy can be protected. It content of search results, but the design of transaction examines monitoring needs of users, systems designers, logging programs has not caught up with these changes. managers, and customer support staff and outlines meaDespite the changing nature of the content of informasures of workload, capacity, and performance for hardware, software, and data communications systems. Fition retrieval systems, the user interface, and the data nally, the article proposes a client-server design for communications interface, a basic research question remonitoring, which involves creation of a series of server mains: Is the system effective in helping users find inforand client systems to obtain and process transaction mation? It is growing more difficult to answer this quesand computer performance information. These systems tion because interface and communications systems are include: A Log Server, which captures all levels of transactions and packets on the network; a Monitor Server, more complex. However, there is much useful informawhich synthesizes the log and packet data; an Assistion ideally suited for logging (such as user actions and tance Server, which processes requests for information system performance) and these logs can be used to supand help from the Web Server in real time; and an Acplement qualitative evaluations or confirm user uncercounting Server, which authenticates user access to the tainty regarding actions taken. system. A special System Administrator Client is proposed to control the monitoring system, as is a System It is extremely expensive to conduct personal interInformation Client to receive real-time and on-demand views with users to find out if they are satisfied with reports of system activity. the information retrieved and/or the information system